Face recognition method, terminal device using the same, and computer readable storage medium

ABSTRACT

A backlight face recognition method, a terminal device using the same, and a computer readable storage medium are provided. The method includes: performing a face detection on each original face image in an original face image sample set to obtain a face frame corresponding to the original face image; capturing the corresponding original face images from the original face image sample set, and obtaining a new face image containing background pixels corresponding to the captured original face images from the original face image sample set; preprocessing all the obtained new face images to obtain a backlight sample set and a normal lighting sample set; and training a convolutional neural network using the backlight sample set and the normal lighting sample set until the convolutional neural network reaches a preset stopping condition. The trained convolutional neural network will improve the accuracy of face recognition in complex background and strong light.

CROSS REFERENCE TO RELATED APPLICATIONS

The present disclosure claims priority to Chinese Patent Application No.202011501362.0, filed Dec. 17, 2020, which is hereby incorporated byreference herein as if set forth in its entirety.

BACKGROUND 1. Technical Field

The present disclosure relates to face recognition technology, andparticularly to a backlight face recognition method, a terminal deviceusing the same, and a computer readable storage medium.

2. Description of Related Art

The face recognition technology is based on human facial features. Forthe input face image or video stream, whether there is a face or not isdetermined first. If there is a face, the position and the size of eachface and the location of each main facial organs are further determined,then the identity features contained in each face are extracted tocompare with known faces, thereby identifying the identity for eachface.

As a new identity authentication technology, the face recognitiontechnology is widely used in terminal devices such as robots, mobilephones, personal digital assistants, and CCTV cameras. When performingface recognition, because the complex background and strong light in theenvironment where the face is located will affect the accuracy of facerecognition, the improvement of the accuracy of face recognition in thecomplex background and strong light had become an urgent problem need tobe solved.

BRIEF DESCRIPTION OF THE DRAWINGS

To describe the technical schemes in the embodiments of the presentdisclosure or in the prior art more clearly, the following brieflyintroduces the drawings required for describing the embodiments or theprior art. It should be noted that, the drawings in the followingdescription merely show some embodiments. For those skilled in the art,other drawings may be obtained according to the drawings withoutcreative efforts.

FIG. 1 is a flow chart of the first embodiment of a backlight facerecognition method according to the present disclosure.

FIG. 2 is a flow chart of the second embodiment of a backlight facerecognition method according to the present disclosure.

FIG. 3 is a schematic diagram of an original face image, a face frame,and a new face image according to an embodiment of the presentdisclosure.

FIG. 4 is a flow chart of the third embodiment of a backlight facerecognition method according to the present disclosure.

FIG. 5 is a schematic diagram of the coordinates of a face frame and aset of its diagonal points as well as the coordinates of a new faceframe and a set of its diagonal points.

FIG. 6 is a flow chart of the fourth embodiment of a backlight facerecognition method according to the present disclosure.

FIG. 7 is a flow chart of the fifth embodiment of a backlight facerecognition method according to the present disclosure.

FIG. 8 is a schematic block diagram of a backlight face recognitionapparatus according to an embodiment of the present disclosure.

FIG. 9 is a schematic block diagram of a terminal device according to anembodiment of the present disclosure.

DETAILED DESCRIPTION

In the following descriptions, for purposes of explanation instead oflimitation, specific details such as particular system architecture andtechnique are set forth in order to provide a thorough understanding ofembodiments of the present disclosure. However, it will be apparent tothose skilled in the art that the present disclosure may be implementedin other embodiments that are less specific of these details. In otherinstances, detailed descriptions of well-known systems, devices,circuits, and methods are omitted so as not to obscure the descriptionof the present disclosure with unnecessary detail.

It is to be understood that, when used in the description and theappended claims of the present disclosure, the terms “including” and“comprising” indicate the presence of stated features, entireties,steps, operations, elements and/or components, but do not preclude thepresence or addition of one or a plurality of other features, integers,steps, operations, elements, components and/or combinations thereof.

It is also to be understood that the term “and/or” used in thedescription and the appended claims of the present disclosure refers toany combination of one or more of the associated listed items and allpossible combinations, and includes such combinations.

As used in the description and the appended claims, the term “if” may beinterpreted as “when” or “once” or “in response to determining” or “inresponse to detecting” according to the context. Similarly, the phrase“if determined” or “if [the described condition or event] is detected”may be interpreted as “once determining” or “in response to determining”or “on detection of [the described condition or event]” or “in responseto detecting [the described condition or event]”.

In addition, in the description and the appended claims of the presentdisclosure, the terms “first”, “second”, “third”, and the like in thedescriptions are only used for distinguishing, and cannot be understoodas indicating or implying relative importance.

In the present disclosure, the descriptions of “one embodiment”, “someembodiments” or the like described in the specification mean that one ormore embodiments of the present disclosure can include particularfeatures, structures, or characteristics which are related to thedescriptions of the descripted embodiments. Therefore, the sentences “inone embodiment”, “in some embodiments”, “in some other embodiments”, “inother embodiments” and the like that appear in different places of thespecification do not mean that descripted embodiments should be referredby all other embodiments, but instead be referred by “one or more butnot all other embodiments” unless otherwise specifically emphasized. Theterms “including”, “comprising”, “having” and their variations all mean“including but not limited to” unless otherwise specifically emphasized.

The backlight face recognition method provided in the embodiments of thepresent disclosure may be applied to a terminal device that has a cameraor can communicate with the camera, for example, a robot, an automatedguided vehicle (AGV), an unmanned aerial vehicle, a mobile phone, asmart ring (e.g., a smart bracelet and a smart neck ring), a tabletcomputer, a laptop computer, a netbook computer, a personal digitalassistant (PDA), a server, and the like, so as to accurately detect theface in the environment, especially the face in complex background orstrong light. The method may be executed by a processor of the terminaldevice by executing a computer program with corresponding functions. Therobot may be a service robot, for example, an educational robot, anentertainment robot, a logistics robot, a nanny robot, a welcome robot,and the like, while the type of the terminal device is not limitedherein.

FIG. 1 is a flow chart of the first embodiment of a backlight facerecognition method according to the present disclosure. A backlight facerecognition method is provided. In one embodiment, the backlight facerecognition method is a computer-implemented method executable for aprocessor, which may be applied to the above-mentioned terminal device.The method may be implemented through a backlight face recognitionapparatus shown in FIG. 8 or a terminal device shown in FIG. 9 . Asshown in FIG. 1 , the method may include the following steps.

S101: performing a face detection on each original face image in anoriginal face image sample set to obtain a face frame corresponding tothe original face image.

In one embodiment, the original face image sample set is an image setcomposed of original face images. The original face image is an imagethat contains background pixels and face pixels that is obtained byphotographing a (human) face through a camera (disposed on the terminaldevice). The background pixels are the pixels corresponding to thebackground objects in the environment where the face is located in theoriginal face image, and the face pixels are the pixels corresponding tothe face in the original face image. In contrary to the backgroundpixels, the face pixels can also be called foreground pixels.

In one embodiment, before step S101, the method may further include:

taking a plurality of original face images through the camera; and

creating the original image sample set including all the original faceimages.

In one embodiment, a face detector called RetinaFace may be used toperform face detection on each original face image, so as to obtain aface frame and face key points corresponding to each original face imageoutput by the face detector. There are usually five face key pointsnamely the center of the left and right eyes, the nose, and the left andright corners of the mouth. The face frame is a rectangular frame thatincludes all face pixels and a few background pixels around the facepixels.

In one embodiment, step S101 may include:

performing the face detection on each original face image in theoriginal face image sample set using the face detector to obtain a faceframe corresponding to the original face image output by the facedetector.

S102: capturing the corresponding original face images from the originalface image sample set based on each of the obtained face frames, andobtaining a new face image containing background pixels corresponding toeach of the captured original face images from the original face imagesample set.

In one embodiment, since the face frame mainly contains face pixelswhile contains a few or even no background pixels, it is necessary tore-capture the original face image according to the face frame so as toobtain the new face image containing more background pixels.

FIG. 2 is a flow chart of the second embodiment of a backlight facerecognition method according to the present disclosure. As shown in FIG.2 , in one embodiment, step S102 may include the following sub-steps:

S201: obtaining a new face frame corresponding to each of the originalface images by expanding each of the face frames; and

S202: capturing the corresponding original face image from the originalface image sample set based on each of the new face frames to obtain thenew face image containing the background pixels corresponding to each ofthe captured original face images from the original face image sampleset.

In one embodiment, after obtaining the face frame, it is expanded toobtain the new face frame, then the new face frame is used to capturethe original face image so as to obtain the new face image containingface pixels and more background pixels. The original face image may becaptured by cropping the area where the new face frame in the originalface image is located, where only the face pixels and background pixelsin the area where the new face frame is located are retained while thebackground pixels outside the new face frame are deleted.

FIG. 3 is a schematic diagram of an original face image, a face frame,and a new face image according to an embodiment of the presentdisclosure. As shown in FIG. 3 , from left to right, there are theoriginal face image (i.e., part (a)), the face frame (i.e., part (b)),and the new face image (i.e., part (c)) in order.

FIG. 4 is a flow chart of the third embodiment of a backlight facerecognition method according to the present disclosure. As shown in FIG.4 , in one embodiment, step S201 may include the following sub-steps:

S401: obtaining coordinates of a set of diagonal points of each of theface frames.

S402: obtaining coordinates of corner points of the new face imagecontaining the background pixels corresponding to each of the capturedoriginal face images by expanding each of the face frames based on thecoordinates of the set of diagonal points of the face frame.

In one embodiment, in the case that the face frame is a rectangularframe, two of its four corner points that are on any side have the sameabscissa (X-axis) or ordinate (Y-axis), and the abscissa and theordinate of each set of diagonal points in the four corner points aredifferent. Therefore, the coordinates of its four corner points can beobtained upon obtaining the coordinates of a set of diagonal points ofthe face frame. After obtaining the coordinates of any set of diagonalpoints of the face frame, the face frame can be expanded based on theobtained coordinates of so that the range of the pixels of the expandednew face frame is larger than the face frame and small than or equal tothe original face image, that is, the minimum value of the abscissas ofthe four corner points of the new face frame should be smaller than theminimum value of the abscissas of the four corner points of the faceframe and larger than or equal to the minimum value of the abscissas ofall the pixels in the original face image, the maximum value of theabscissas of the four corner points of the new face frame should belarger than the maximum value of the abscissas of the four corner pointsof the face frame and smaller than or equal to the minimum value of theabscissas of all the pixels in the original face image, the minimumvalue of the ordinate of the four corner points of the new face frameshould be smaller than the minimum value of the ordinate of the fourcorner points of the face frame and larger than or equal to the minimumvalue of the ordinate of all the pixels in the original face image, themaximum value of the ordinate of the four corner points of the new faceframe should be larger than the maximum value of the ordinate of thefour corner points of the face frame and smaller than or equal to theminimum value of the abscissas of all the pixels in the original faceimage.

In one embodiment, step S402 may include the following sub-steps:

obtaining coordinates of a center point of each of the face frames basedon the coordinates of the set of diagonal points of the face frame; and

obtaining the coordinates of the corner points of the new face imagecontaining the background pixels corresponding to each of the capturedoriginal face images based on a preset expansion ratio coefficient andthe coordinates of the set of diagonal points and the coordinates of thecenter point of each of the face frames.

In one embodiment, the new face frame may be obtained by taking thecenter point of the face frame as the origin to expand toward thepositive and negative directions of the abscissa of the face frame andthe positive and negative directions of the ordinate of the face frame,that is, taking the center point of the face frame as the origin toexpand toward the periphery of the face frame. The preset expansionratio coefficient may be set according to actual needs, which should belarger than the face frame and smaller than the original face image. Theabscissa and the ordinate of the new face frame may correspond todifferent preset expansion ratio coefficients, that is, the expansionratios of the abscissa and the ordinate may be different.

In one embodiment, the coordinates of the center point of each of theface frames may be calculated through equations of:x _(center)=(x ₂ −x ₁)/2; andy _(center)=(y ₂ −y ₁)/2;

where, x_(center) represents the abscissa of the center point of any ofthe face frames, y_(center) represents the ordinate of the center pointof the face frame, x₁ represents the abscissa of a first diagonal pointin the set of diagonal points of the face frame, x₂ represents theabscissa of a second diagonal point in the set of diagonal points of theface frame, y₁ represents the ordinate of the first diagonal point inthe set of diagonal points of the face frame, and y₂ represents theordinate of the second diagonal point in the set of diagonal points ofthe face frame; and

the coordinates of the corner points of the new face image containingthe background pixels that corresponds to each of the captured originalface images may be calculated through equations of:max_long=max((x ₂ −x _(i)),(y ₂ −y ₁));x _(new1) =x _(center)−(max_long*α₁)/2;x _(new2) =x _(center)+(max_long*α₁)/2;y _(new1) =y _(center)−(max_long*α₂)/2; andy _(new2) =y _(center)+(max_long*α₂)/2;

where, max represents the maximum value function, x_(new1) representsthe abscissa of a first diagonal point in a set of diagonal points ofany new face frame, x_(new2) represents the abscissa of a seconddiagonal point in the set of diagonal points of any new face frame,y_(new1) represents the ordinate of the first diagonal point in the setof diagonal points of any new face frame, y_(new2) the ordinate of thesecond diagonal point in the set of diagonal points of any new faceframe, at represents a first preset expansion ratio coefficient, and α₂represents a second preset expansion ratio coefficient.

In one embodiment, the first preset expansion ratio coefficient and thesecond preset expansion ratio coefficient may be set to be the same ordifferent according to actual needs. In the case that the new face frameis a rectangular frame, two of its four corner points that are on anyside have the same abscissa (X-axis) or ordinate (Y-axis), and theabscissa and the ordinate of each set of diagonal points in the fourcorner points are different. Therefore, the coordinates of its fourcorner points can be obtained upon obtaining the coordinates of any setof diagonal points of the new face frame.

FIG. 5 is a schematic diagram of the coordinates of a face frame and aset of its diagonal points as well as the coordinates of a new faceframe and a set of its diagonal points. Referring to FIG. 5 , thecoordinates of the face frame 51 and a set of its diagonal points are(x₁, y₁) and (x₂, y₂), respectively, and the coordinates of the faceframe 52 and a set of its diagonal points are (x_(new1), y_(new1)) and(x_(new2), y_(new2)), respectively.

S103: preprocessing all the obtained new face images to obtain abacklight sample set and a normal lighting sample set.

In one embodiment, in order to enable the convolutional neural networkto detect both the face in complex background and that in strong light,after capturing the original face image to obtain the new face imagecontaining the background pixels, it further needs to preprocess the newface image so as to obtain the backlight sample set composed ofbacklight samples and the normal lighting sample set composed of normallighting samples. In which, the backlight sample is the new face imagewith at least one of missing facial feature and blurred facial contourfeature, and the normal lighting sample is the new face image withcomplete facial feature and clear facial contour feature.

In one embodiment, the convolutional neural network may be a lightweightconvolutional neural network, for example, the shufflenetV2 0.5×[2]model or the shufflenetV2 0.25× model. The accuracy of the shufflenetV20.5×[2] model for detecting faces is 96%, where the accuracy isrelatively higher and the maximum image size that can be input is224*224. The accuracy of the shufflenetV2 0.25× model for detectingfaces is 90%, where the accuracy is lower than that of the shufflenetV20.5×[2] model while the number of the output channels is relativelysmall and the calculation amount of is relatively small.

In one embodiment, before step S103, the method may further include:

reducing each of the new face images according to a preset reductionratio.

In one embodiment, the size of each new face image may be reduced,thereby reducing the calculation amount of the convolutional neuralnetwork. The preset reduction ratio may be set according to actualneeds, as long as the size of the reduced new face image is larger thanor equal to the minimum image size that can be recognized by theconvolutional neural network. For example, when the shufflenetV2 0.5×[2]model is used, the size of each new face image can be reduced to128*128, that is, the preset reduction ratio is (128/224)*(128/224)=1/3,so that the calculation amount of the shufflenetV20.5×[2] model can bereduced to ⅓ of the original.

FIG. 6 is a flow chart of the fourth embodiment of a backlight facerecognition method according to the present disclosure. As shown in FIG.6 , in one embodiment, step S103 may include the following sub-steps:

S601: classifying each of the new face images as one of a backlightsample and a normal lighting sample;

S602: performing a geometric transformation on each classified backlightsample to obtain at least a preprocessed backlight sample correspondingto the classified backlight sample; and

S603: creating the backlight sample set including all the classifiedbacklight samples and all the obtained preprocessed backlight samples,and creating the normal lighting sample set including all the normallighting samples.

In one embodiment, each new face image is classified according to thecompleteness of facial features and the clarity of facial contourfeatures in the new face image first, then the new facial images withmissing facial feature or blurred facial feature contours feature isclassified as the backlight sample and added to the backlight sample setand the new facial images with complete facial feature and clear facialcontour feature is classified as the normal lighting sample and added tothe normal lighting sample set.

In one embodiment, since the number of backlight samples is usually lessthan that of normal lighting samples, geometric transformation such asmirror flip, rotation, translation, zooming and the like may beperformed on each backlight sample to obtain a pre-processed backlightsample corresponding to the backlight sample to add to the backlightsample set so as to enrich the backlight sample set, so that the numberof samples in the backlight sample set and the normal lighting sampleset can be balanced.

S104: training a convolutional neural network using the backlight sampleset and the normal lighting sample set until the convolutional neuralnetwork reaches a preset stopping condition.

In one embodiment, after creating the backlight sample set and thenormal lighting sample set, they can be used to train the convolutionalneural network until the convolutional neural network reaches the presetstopping condition. The preset stopping condition may be set accordingto actual needs, which may be, for example, the accuracy of theconvolutional neural network to classify backlight samples or normallighting samples being greater than a preset accuracy threshold, theconverges and loss functions of the convolutional neural network beingreduced to a preset loss value, the performance of the convolutionalneural network being no longer improved, and the like. The case that theperformance of the convolutional neural network is no longer improvedmay be its accuracy of classifying backlight samples or normal lightingsamples being no longer improved or the loss being no longer reduced.The preset accuracy threshold and the preset loss value may be setaccording to actual needs. For example, the preset accuracy thresholdmay be any value between 93% and 99% such as 96%.

FIG. 7 is a flow chart of the fifth embodiment of a backlight facerecognition method according to the present disclosure. As shown in FIG.7 , in one embodiment, step S104 may include the following sub-steps:

S701: training the convolutional neural network through the firstbacklight sample subset and the first normal lighting sample subset toobtain the trained convolutional neural network;

S702: classifying the second backlight sample subset and the secondnormal lighting sample subset using the trained convolutional neuralnetwork to obtain a classification result;

S703: adding one or more incorrectly classified second backlight samplesin the classification result to the first backlight sample subset, andadding one or more incorrectly classified second normal lighting samplesin the classification result to the first normal lighting sample subset;and

S704: returning to the training the convolutional neural network throughthe first backlight sample subset and the first normal lighting samplesubset to obtain the trained convolutional neural network until theconvolutional neural network reaches the preset stopping condition.

In one embodiment, the backlight sample set and the normal lightingsample set may be respectively divided into two subsets, then the firstsubset may be used to train the convolutional neural network, and thenthe second subset may be used to test the trained convolutional neuralnetwork. The trained convolutional neural network is used to classifythe samples in the second subset to obtain the classification result.Based on the classification result, the backlight samples in theclassification result that are incorrectly classified as the normallighting samples and the normal lighting samples in the classificationresult that are incorrectly classified as the backlight samples areadded to the first subset, and the first subset is used to train thetrained convolutional neural network again. The forgoing process will beperformed in a loop manner until the convolutional neural networkreaches the preset stopping condition.

In one embodiment, the sample set used to train the convolutional neuralnetwork may also be the backlight sample set and the normal lightingsample set. The sample set used to test the trained convolutional neuralnetwork each time to obtain the classification result may also be thenew backlight sample set and the new normal lighting sample set. The newbacklight sample set and the new normal lighting sample set are createdusing the new original faces in the same way as the backlight sample setand the normal lighting sample set.

In one embodiment, after step S104, the method may further include:

collecting face image(s) through a camera; and

classifying the collected face image(s) through the trainedconvolutional neural network to obtain a classification result.

In one embodiment, after the convolutional neural network is trained toreach the preset stopping condition, it can be used to classify any faceimage. The convolutional neural network that is trained to reach thepreset stopping condition can accurately distinguish the face in thebacklight and the face in the normal lighting, which can improve theeffect of the face recognition and the accuracy of the input offace-based identification information, thereby improving the accuracy ofthe face recognition of the terminal device using the backlight facerecognition method so as to enhance user experience and meet user needs.

In the backlight face recognition method, the convolutional neuralnetwork is trained using the backlight sample set and the normallighting sample set composed of the new face image containing thebackground pixels, which can effectively improve the accuracy of theface recognition for a terminal device that is realized based on theconvolutional neural network in complex background and strong light.

It should be understood that, the sequence of the serial number of thesteps in the above-mentioned embodiments does not mean the executionorder while the execution order of each process should be determined byits function and internal logic, which should not be taken as anylimitation to the implementation process of the embodiments.

FIG. 8 is a schematic block diagram of a face recognition apparatusaccording to an embodiment of the present disclosure. A face recognitionapparatus 100 for performing the steps in the above-mentioned backlightface recognition method is provided. The backlight face recognitionapparatus may be a virtual appliance in the above-mentioned terminaldevice that is executed by the processor of the terminal device, or bethe terminal device itself.

As shown in FIG. 8 , the face recognition apparatus 100 may include:

a face detection unit 101 configured to perform a face detection on eachoriginal face image in an original face image sample set to obtain aface frame corresponding to the original face image;

an image capturing unit 102 configured to the corresponding originalface images from the original face image sample set based on each of theobtained face frames, and obtain a new face image containing backgroundpixels corresponding to the captured original face images from theoriginal face image sample set;

an image preprocessing unit 103 configured to preprocess all theobtained new face images to obtain a backlight sample set and a normallighting sample set; and

a network training unit 104 configured to train a convolutional neuralnetwork using the backlight sample set and the normal lighting sampleset until the convolutional neural network reaches a preset stoppingcondition.

In one embodiment, the face recognition apparatus 100 may furtherinclude:

an image zooming unit configured to reduce each of the new face imagesaccording to a preset reduction ratio.

In one embodiment, the face recognition apparatus 100 may furtherinclude:

an image collection unit configured to collect a face image through acamera; and

an image classification unit configured to classify the collected faceimage through the trained convolutional neural network to obtain aclassification result.

In one embodiment, each unit in the face recognition apparatus 100 maybe a software unit, or be implemented by logic circuits integrated inthe processor, or may be implemented by a plurality of distributedprocessors.

FIG. 9 is a schematic block diagram of a terminal device according to anembodiment of the present disclosure. As shown in FIG. 9 , a terminaldevice 200 is provided, which includes: at least one processor 201 (onlyone processor is shown in FIG. 9 ), a storage 202, a computer program203 stored in the storage 202 and executable on the processor 201, and acamera 204 connected to the processor 201. The processor 201 implementsthe steps in any of the foregoing method embodiments when executing thecomputer program 203.

Those skilled in the art can understand that, FIG. 9 is only an exampleof a terminal device, and does not form a limitation on the terminaldevice 200. It may include more or less components than shown in thefigure, some of the components may be combined, or may include othercomponents, for example, including input device and output devices,network access devices, or the like.

In one embodiment, the processor 201 may be a central processing unit(CPU), or be other general purpose processor, a digital signal processor(DSP), an application specific integrated circuit (ASIC), afield-programmable gate array (FPGA), or be other programmable logicdevice, a discrete gate, a transistor logic device, and a discretehardware component. The general purpose processor may be amicroprocessor, or the processor may also be any conventional processor.

In one embodiment, the storage 201 may be an internal storage unit ofthe terminal device 200, for example, a hard disk or a memory of theterminal device 200. The storage 202 may also be an external storagedevice of the terminal device 200, for example, a plug-in hard disk, asmart media card (SMC), a secure digital (SD) card, flash card, and thelike, which is equipped on the terminal device 200. Furthermore, thestorage 202 may further include both an internal storage unit and anexternal storage device, of the terminal device 200. The storage 202 isconfigured to store the operating system, application programs, a bootloader, and the related data. The storage 202 may also be used totemporarily store the data that has been output or will be output.

It should be noted that, the information exchange and execution processbetween the above-mentioned apparatus/units are based on the sameconcept as the above-mentioned method embodiments. The specificfunctions and technical effects are as the above-mentioned methodembodiments, and are not described herein.

Those skilled in the art may clearly understand that, for theconvenience and simplicity of description, the division of theabove-mentioned functional units is merely an example for illustration.In actual applications, the above-mentioned functions may be allocatedto be performed by different functional units according to requirements,that is, the internal structure of the device may be divided intodifferent functional units or modules to complete all or part of theabove-mentioned functions. The functional units in the embodiments maybe integrated in one processing unit, or each unit may exist alonephysically, or two or more units may be integrated in one unit. Theabove-mentioned integrated unit may be implemented in the form ofhardware or in the form of software functional unit. In addition, thespecific name of each functional unit is merely for the convenience ofdistinguishing each other and are not intended to limit the scope ofprotection of the present disclosure. For the specific operation processof the units in the above-mentioned system, reference may be made to thecorresponding processes in the above-mentioned method embodiments, andare not described herein.

In the embodiments of the present disclosure, a network device isfurther provided, which includes at least one processor, a storage, anda computer program stored in the storage executable on the processor.The processor implements the steps in the above-mentioned methodembodiments when executing the computer program.

In the embodiments of the present disclosure, a non-transitorycomputer-readable storage medium is further provided, which stores acomputer program. When the computer program is executed by a processor,the steps in the above-mentioned method embodiments is implemented.

In the embodiments of the present disclosure, a computer program productis further provided. When the computer program product is executed on aterminal device, the steps in the above-mentioned method embodiments isimplemented.

When the integrated unit is implemented in the form of a softwarefunctional unit and is sold or used as an independent product, theintegrated module/unit may be stored in a non-transitory computerreadable storage medium. Based on this understanding, all or part of theprocesses in the method for implementing the above-mentioned embodimentsof the present disclosure are implemented, and may also be implementedby instructing relevant hardware through a computer program. Thecomputer program may be stored in a non-transitory computer readablestorage medium, which may implement the steps of each of theabove-mentioned method embodiments when executed by a processor. Inwhich, the computer program includes computer program codes which may bethe form of source codes, object codes, executable files, certainintermediate, and the like. The computer readable medium may include atleast any primitive or device capable of carrying the computer programcodes to an apparatus/device, a recording medium, a computer memory, aread-only memory (ROM), a random access memory (RAM), electric carriersignals, and telecommunication signals and software distribution media,for example, a USB flash drive, a portable hard disk, a magnetic disk,and an optical disk, It should be noted that the content contained inthe computer readable medium may be appropriately increased or decreasedaccording to the requirements of legislation and patent practice in thejurisdiction. For example, in some jurisdictions, according to thelegislation and patent practice, a computer readable medium does notinclude electric carrier signals and telecommunication signals.

In the above-mentioned embodiments, the description of each embodimenthas its focuses, and the parts which are not described or mentioned inone embodiment may refer to the related descriptions in otherembodiments.

Those ordinary skilled in the art may clearly understand that, theexemplificative units and steps described in the embodiments disclosedherein may be implemented through electronic hardware or a combinationof computer software and electronic hardware. Whether these functionsare implemented through hardware or software depends on the specificapplication and design constraints of the technical schemes. Thoseordinary skilled in the art may implement the described functions indifferent manners for each particular application, while suchimplementation should not be considered as beyond the scope of thepresent disclosure.

In the embodiments provided by the present disclosure, it should beunderstood that the disclosed apparatus/device and method may beimplemented in other manners. For example, the above-mentionedapparatus/device embodiment is merely exemplary. For example, thedivision of modules or units is merely a logical functional division,and other division manner may be used in actual implementations, thatis, multiple units or components may be combined or be integrated intoanother system, or some of the features may be ignored or not performed.In addition, the shown or discussed mutual coupling may be directcoupling or communication connection, and may also be indirect couplingor communication connection through some interfaces, devices or units,and may also be electrical, mechanical or other forms.

The units described as separate components may or may not be physicallyseparated. The components represented as units may or may not bephysical units, that is, may be located in one place or be distributedto multiple network units. Some or all of the units may be selectedaccording to actual needs to achieve the objectives of this embodiment.

The above-mentioned embodiments are merely intended for describing butnot for limiting the technical schemes of the present disclosure.Although the present disclosure is described in detail with reference tothe above-mentioned embodiments, it should be understood by thoseskilled in the art that, the technical schemes in each of theabove-mentioned embodiments may still be modified, or some of thetechnical features may be equivalently replaced, while thesemodifications or replacements do not make the essence of thecorresponding technical schemes depart from the spirit and scope of thetechnical schemes of each of the embodiments of the present disclosure,and should be included within the scope of the present disclosure.

What is claimed is:
 1. A computer-implemented backlight face recognitionmethod, comprising: performing a face detection on each original faceimage in an original face image sample set to obtain a face framecorresponding to the original face image; capturing the correspondingoriginal face images from the original face image sample set based oneach of the obtained face frames, and obtaining a new face imagecontaining background pixels corresponding to each of the capturedoriginal face images from the original face image sample set;preprocessing all the obtained new face images to obtain a backlightsample set and a normal lighting sample set; and training aconvolutional neural network using the backlight sample set and thenormal lighting sample set until the convolutional neural networkreaches a preset stopping condition.
 2. The method of claim 1, whereinthe capturing the corresponding original face images from the originalface image sample set based on each of the obtained face frames, andobtaining the new face image containing the background pixelscorresponding to each of the captured original face images from theoriginal face image sample set comprises: obtaining a new face framecorresponding to each of the original face images by expanding each ofthe face frames; and capturing the corresponding original face imagesfrom the original face image sample set based on each of the new faceframes to obtain the new face image containing the background pixelscorresponding to each of the captured original face images from theoriginal face image sample set.
 3. The method of claim 2, wherein theobtaining the new face frame corresponding to each of the original faceimages by expanding each of the face frames comprises: obtainingcoordinates of a set of diagonal points of each of the face frames; andobtaining coordinates of corner points of the new face image containingthe background pixels corresponding to each of the captured originalface images by expanding each of the face frames based on thecoordinates of the set of diagonal points of the face frame.
 4. Themethod of claim 3, wherein the obtaining the coordinates of the cornerpoints of the new face image containing the background pixelscorresponding to each of the captured original face images by expandingeach of the face frames based on the coordinates of the set of diagonalpoints of the face frame comprises: obtaining coordinates of a centerpoint of each of the face frames based on the coordinates of the set ofdiagonal points of the face frame; and obtaining the coordinates of thecorner points of the new face image containing the background pixelscorresponding to each of the captured original face images based on apreset expansion ratio coefficient and the coordinates of the set ofdiagonal points and the coordinates of the center point of each of theface frames.
 5. The method of claim 4, wherein the coordinates of thecenter point of each of the face frames is calculated through equationsof:x _(center)=(x ₂ −x ₁)/2; andy _(center)=(y ₂ −y ₁)/2; where, x_(center) represents the abscissa of acenter point of any of the face frames, y_(center) represents theordinate of the center point of the face frame, x₁ represents theabscissa of a first diagonal point in the set of diagonal points of theface frame, x₂ represents the abscissa of a second diagonal point in theset of diagonal points of the face frame, y₁ represents the ordinate ofthe first diagonal point in the set of diagonal points of the faceframe, and y₂ represents the ordinate of the second diagonal point inthe set of diagonal points of the face frame; the coordinates of thecorner points of the new face image containing the background pixelscorresponding to each of the captured original face images is calculatedthrough equations of:max_long=max((x ₂ −x _(i)),(y ₂ −y ₁));x _(new1) =x _(center)−(max_long*α₁)/2;x _(new2) =x _(center)+(max_long*α₁)/2;y _(new1) =y _(center)−(max_long*α₂)/2; andy _(new2) =y _(center)+(max_long*α₂)/2; where, max represents a maximumvalue function, x_(new1) represents the abscissa of a first diagonalpoint in a set of diagonal points of any new face frame, x_(new2)represents the abscissa of a second diagonal point in the set ofdiagonal points of any new face frame, y_(new1) represents the ordinateof the first diagonal point in the set of diagonal points of any newface frame, y_(new2) the ordinate of the second diagonal point in theset of diagonal points of any new face frame, α₁ represents a firstpreset expansion ratio coefficient, and α₂ represents a second presetexpansion ratio coefficient.
 6. The method of claim 1, wherein thepreprocessing all the obtained new face images to obtain the backlightsample set and the normal lighting sample set comprises: classifyingeach of the new face images as one of a backlight sample and a normallighting sample; wherein the backlight sample is the new face image withat least one of missing facial feature and blurred facial contourfeature, and the normal lighting sample is the new face image withcomplete facial feature and clear facial contour feature; performing ageometric transformation on each classified backlight sample to obtainat least a preprocessed backlight sample corresponding to the classifiedbacklight sample; and creating the backlight sample set including allthe classified backlight samples and all the obtained preprocessedbacklight samples, and creating the normal lighting sample set includingall normal lighting samples.
 7. The method of claim 6, wherein beforethe preprocessing all the obtained new face images to obtain thebacklight sample set and the normal lighting sample set, the methodfurther comprises: reducing each of the new face images according to apreset reduction ratio.
 8. The method of claim 1, wherein the backlightsample set includes a first backlight sample subset and a secondbacklight sample subset, and the normal lighting sample set includes afirst normal lighting sample subset and a second normal lighting samplesubset; the training the convolutional neural network using thebacklight sample set and the normal lighting sample set until theconvolutional neural network reaches the preset stopping conditioncomprises: training the convolutional neural network through the firstbacklight sample subset and the first normal lighting sample subset toobtain the trained convolutional neural network; classifying the secondbacklight sample subset and the second normal lighting sample subsetusing the trained convolutional neural network to obtain aclassification result; adding one or more incorrectly classified secondbacklight samples in the classification result to the first backlightsample subset, and adding one or more incorrectly classified secondnormal lighting samples in the classification result to the first normallighting sample subset; and returning to the training the convolutionalneural network through the first backlight sample subset and the firstnormal lighting sample subset to obtain the trained convolutional neuralnetwork until the convolutional neural network reaches the presetstopping condition.
 9. A terminal device, comprising: a processor; amemory coupled to the processor; and one or more computer programsstored in the memory and executable on the processor; wherein, the oneor more computer programs comprise: instructions for performing a facedetection on each original face image in an original face image sampleset to obtain a face frame corresponding to the original face image;instructions for capturing the corresponding original face images fromthe original face image sample set based on each of the obtained faceframes, and obtaining a new face image containing background pixelscorresponding to each of the captured original face images from theoriginal face image sample set; instructions for preprocessing all theobtained new face images to obtain a backlight sample set and a normallighting sample set; and instructions for training a convolutionalneural network using the backlight sample set and the normal lightingsample set until the convolutional neural network reaches a presetstopping condition.
 10. The terminal device of claim 9, wherein theinstructions for capturing the corresponding original face images fromthe original face image sample set based on each of the obtained faceframes, and obtaining the new face image containing the backgroundpixels corresponding to each of the captured original face images fromthe original face image sample set comprise: instructions for obtaininga new face frame corresponding to each of the original face images byexpanding each of the face frames; and instructions for capturing thecorresponding original face images from the original face image sampleset based on each of the new face frames to obtain the new face imagecontaining the background pixels corresponding to each of the capturedoriginal face images from the original face image sample set.
 11. Theterminal device of claim 10, wherein the instructions for obtaining thenew face frame corresponding to each of the original face images byexpanding each of the face frames comprise: instructions for obtainingcoordinates of a set of diagonal points of each of the face frames; andinstructions for obtaining coordinates of corner points of the new faceimage containing the background pixels corresponding to each of thecaptured original face images by expanding each of the face frames basedon the coordinates of the set of diagonal points of the face frame. 12.The terminal device of claim 11, wherein the instructions for obtainingthe coordinates of the corner points of the new face image containingthe background pixels corresponding to each of the captured originalface images by expanding each of the face frames based on thecoordinates of the set of diagonal points of the face frame comprise:instructions for obtaining coordinates of a center point of each of theface frames based on the coordinates of the set of diagonal points ofthe face frame; and instructions for obtaining the coordinates of thecorner points of the new face image containing the background pixelscorresponding to each of the captured original face images based on apreset expansion ratio coefficient and the coordinates of the set ofdiagonal points and the coordinates of the center point of each of theface frames.
 13. The terminal device of claim 12, wherein thecoordinates of the center point of each of the face frames is calculatedthrough equations of:x _(center)=(x ₂ −x ₁)/2; andy _(center)=(y ₂ −y ₁)/2; where, x_(center) represents the abscissa of acenter point of any of the face frames, y_(center) represents theordinate of the center point of the face frame, x₁ represents theabscissa of a first diagonal point in the set of diagonal points of theface frame, x₂ represents the abscissa of a second diagonal point in theset of diagonal points of the face frame, y₁ represents the ordinate ofthe first diagonal point in the set of diagonal points of the faceframe, and y₂ represents the ordinate of the second diagonal point inthe set of diagonal points of the face frame; the coordinates of thecorner points of the new face image containing the background pixelscorresponding to each of the captured original face images is calculatedthrough equations of:max_long=max((x ₂ −x _(i)),(y ₂ −y ₁));x _(new1) =x _(center)−(max_long*α₁)/2;x _(new2) =x _(center)+(max_long*α₁)/2;y _(new1) =y _(center)−(max_long*α₂)/2; andy _(new2) =y _(center)+(max_long*α₂)/2; where, max represents a maximumvalue function, x_(new1) represents the abscissa of a first diagonalpoint in a set of diagonal points of any new face frame, x_(new2)represents the abscissa of a second diagonal point in the set ofdiagonal points of any new face frame, y_(new1) represents the ordinateof the first diagonal point in the set of diagonal points of any newface frame, y_(new2) the ordinate of the second diagonal point in theset of diagonal points of any new face frame, α₁ represents a firstpreset expansion ratio coefficient, and α₂ represents a second presetexpansion ratio coefficient.
 14. The terminal device of claim 9, whereinthe instructions for preprocessing all the obtained new face images toobtain the backlight sample set and the normal lighting sample setcomprise: instructions for classifying each of the new face images asone of a backlight sample and a normal lighting sample; wherein thebacklight sample is the new face image with at least one of missingfacial feature and blurred facial contour feature, and the normallighting sample is the new face image with complete facial feature andclear facial contour feature; instructions for performing a geometrictransformation on each classified backlight sample to obtain at least apreprocessed backlight sample corresponding to the classified backlightsample; and instructions for creating the backlight sample set includingall the classified backlight samples and all the obtained preprocessedbacklight samples, and creating the normal lighting sample set includingall normal lighting samples.
 15. The terminal device of claim 14,wherein the one or more computer programs further comprise: instructionsfor reducing each of the new face images according to a preset reductionratio.
 16. The terminal device of claim 9, wherein the backlight sampleset includes a first backlight sample subset and a second backlightsample subset, and the normal lighting sample set includes a firstnormal lighting sample subset and a second normal lighting samplesubset; the instructions for training the convolutional neural networkusing the backlight sample set and the normal lighting sample set untilthe convolutional neural network reaches the preset stopping conditioncomprise: instructions for training the convolutional neural networkthrough the first backlight sample subset and the first normal lightingsample subset to obtain the trained convolutional neural network;instructions for classifying the second backlight sample subset and thesecond normal lighting sample subset using the trained convolutionalneural network to obtain a classification result; instructions foradding one or more incorrectly classified second backlight samples inthe classification result to the first backlight sample subset, andadding one or more incorrectly classified second normal lighting samplesin the classification result to the first normal lighting sample subset;and instructions for returning to the training the convolutional neuralnetwork through the first backlight sample subset and the first normallighting sample subset to obtain the trained convolutional neuralnetwork until the convolutional neural network reaches the presetstopping condition.
 17. A non-transitory computer readable storagemedium for storing one or more computer programs, wherein the one ormore computer programs comprise: instructions for performing a facedetection on each original face image in an original face image sampleset to obtain a face frame corresponding to the original face image;instructions for capturing the corresponding original face images fromthe original face image sample set based on each of the obtained faceframes, and obtaining a new face image containing background pixelscorresponding to each of the captured original face images from theoriginal face image sample set; instructions for preprocessing all theobtained new face images to obtain a backlight sample set and a normallighting sample set; and instructions for training a convolutionalneural network using the backlight sample set and the normal lightingsample set until the convolutional neural network reaches a presetstopping condition.
 18. The non-transitory computer readable storagemedium of claim 17, wherein the instructions for capturing thecorresponding original face images from the original face image sampleset based on each of the obtained face frames, and obtaining the newface image containing the background pixels corresponding to each of thecaptured original face images from the original face image sample setcomprise: instructions for obtaining a new face frame corresponding toeach of the original face images by expanding each of the face frames;and instructions for capturing the corresponding original face imagesfrom the original face image sample set based on each of the new faceframes to obtain the new face image containing the background pixelscorresponding to each of the captured original face images from theoriginal face image sample set.
 19. The non-transitory computer readablestorage medium of claim 18, wherein the instructions for obtaining thenew face frame corresponding to each of the original face images byexpanding each of the face frames comprise: instructions for obtainingcoordinates of a set of diagonal points of each of the face frames; andinstructions for obtaining coordinates of corner points of the new faceimage containing the background pixels corresponding to each of thecaptured original face images by expanding each of the face frames basedon the coordinates of the set of diagonal points of the face frame. 20.The non-transitory computer readable storage medium of claim 19, whereinthe instructions for obtaining the coordinates of the corner points ofthe new face image containing the background pixels corresponding toeach of the captured original face images by expanding each of the faceframes based on the coordinates of the set of diagonal points of theface frame comprise: instructions for obtaining coordinates of a centerpoint of each of the face frames based on the coordinates of the set ofdiagonal points of the face frame; and instructions for obtaining thecoordinates of the corner points of the new face image containing thebackground pixels corresponding to each of the captured original faceimages based on a preset expansion ratio coefficient and the coordinatesof the set of diagonal points and the coordinates of the center point ofeach of the face frames.